Skip to content

Add frontend, more granulare cachine#47

Open
adelavega wants to merge 5 commits into
masterfrom
enh/frontend
Open

Add frontend, more granulare cachine#47
adelavega wants to merge 5 commits into
masterfrom
enh/frontend

Conversation

@adelavega
Copy link
Copy Markdown
Member

This PR does two major things:

  1. Adds a MVP of frontend UI for autonima
    • UI is launched from CLI
    • Streamlines specification of a project config
    • Handles "projects" and versions of configs within a project
  2. Refactored caching
    • Caching now adds a signature to the cached data, to ensure incorrect caches are not re-used. This is granular and on a study and stage level. If a full text of a study (or other metadata) change, study is re-run
    • Similarly, if any config parameters for a given stage changes, the stage/studies in that stage are re-run
    • Importantly, early stage changes don't automatically invalidate subsequent caches, as studies may still be passed forward that are previously cached. But studies are filtered using new inclusion lists to ensure incorrect studies are not passed forward

adelavega added 3 commits May 18, 2026 15:54
Added execution provenance utilities in execution.py:deterministic config/stage hashing
config.executed.yaml
execution_manifest.json
cache preview/invalidation
valid-cache copy-forward support

Updated pipeline execution in pipeline.py:writes manifests at start/end/failure
embeds stage cache signatures in output files
invalidates stale stage artifacts under cache_policy=auto

Added CLI flags:--cache-policy auto|ignore|trust-legacy
--clear-cache abstract|fulltext|parsing|annotation|...
--copy-valid-cache-from PATH

Added per-study screening cache signatures in screener.py.
Tightened PubMed cache reuse so cached search results only return PMIDs from the current search/list.
Prevented stale annotation decisions from leaking forward by filtering cached annotation results to the current eligibility graph.
Added UI/backend run options:default UI behavior branches to a new execution folder when spec signatures changed
valid cache is copied forward from the previous output
advanced cache controls in the Runs tab

Added tests in test_execution.py.
@adelavega
Copy link
Copy Markdown
Member Author

The UI/orchestrator works as follows:

  • On each run, Autonima computes deterministic hashes from the normalized config/schema, including per-stage hashes for search, screening, retrieval, parsing, annotation, and output.
  • These hashes are provenance/cache-validity metadata. They describe the execution conditions, not the final result contents.
  • If the spec has not changed, the UI reuses the latest compatible output folder and valid cached artifacts. In that case, no new execution folder is required.
  • If the spec changes in a way that invalidates one or more stages, the UI creates a new timestamped execution folder.
  • When a new execution folder is created, Autonima copies forward only cache artifacts that are still valid for the new config.
  • Invalidated stages are rerun; valid upstream/downstream artifacts may be reused when safe.
  • Each output folder stores the exact executed config snapshot and execution manifest for provenance.

For meta-analysis:

  • You select a specific NIMADS source, which corresponds to a completed screening output folder containing outputs/nimads_studyset.json.
  • The meta-analysis is run using that selected screening output.
  • The resulting meta-analysis artifacts are associated with that selected NIMADS source, so different screening outputs can have separate meta-analysis results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant